© John Wiley & Sons, Inc.

FIGURE 18-6: The classification table for the radiation example.

Overall accuracy: This refers to the proportion of accurate predictions, as shown in the

concordant cells, which are the upper-left and lower-right. Of the 30 individuals in the data set

from Table 18-1, the logistic model predicted correctly

= 0.87, or about 87 percent

of the time. This means with the cut value where you placed it, the model would make a wrong

prediction only about 13 percent of the time.

Sensitivity: This refers to the proportion of yes outcomes predicted accurately. As seen in the

upper-left cell in Figure 18-6, with the cut value where it was placed, the logistic model predicted

13 of the 15 observed deaths (yes outcomes). So the sensitivity is

= 0.87, or about 87

percent. This means the model would have a false-negative rate of 13 percent.

Specificity: This refers to the proportion of no outcomes predicted accurately. In the lower-right

cell of Figure 18-6, the model predicted survival in 13 of the 15 observed survivors. So, the

specificity is

= 0.87, or about 87 percent. This means the model would have a false-

positive rate of 13 percent.

Sensitivity and specificity are especially relevant to screening tests for diseases. An ideal test would

have 100 percent sensitivity and 100 percent specificity, and therefore, 100 percent overall accuracy.

In reality, no test could meet these standards, and there is a tradeoff between sensitivity and specificity.

By judiciously choosing the cut value for converting a predicted probability into a yes or no

decision, you can often achieve high sensitivity or high specificity, but it’s hard to maximize both

simultaneously. Screening tests are meant to detect disease, so how you select the cut value

depends upon what happens if it produces a false-positive or false-negative result. This helps you

decide whether to prioritize sensitivity or specificity.

The sensitivity and specificity of a logistic model depends upon the cut value you set for the

predicted probability. The trick is to select a cut value that gives the optimal combination of

sensitivity and specificity, striking the best balance between false-positive and false-negative

predictions, in light of the different consequences of the two types of false predictions. A false-

positive screening result from a mammogram may mean the patient is worried until the negative

diagnosis is confirmed by ultrasound, and a false-negative screening results from a prostate

cancer screening may result in a delay in identifying the prostate tumor. To find this optimal cut

value, you need to know precisely how sensitivity and specificity play against each other — that

is, how they simultaneously vary with different cut values. There’s a neat way to do that which

we explain in the following section.

Rocking with ROC curves

The graph used to display the sensitivity/specificity tradeoff for any fitted logistic model is called the

Receiver Operator Characteristics (ROC) graph. The name comes from its original use during World